Removed switchToModelVersionAt from SO API definition#219029
Removed switchToModelVersionAt from SO API definition#219029TinaHeiligers merged 10 commits intoelastic:mainfrom
Conversation
…ject type registration definitions
…ModelVersionAt from the public SO type
|
@elasticmachine merge upstream |
|
Pinging @elastic/kibana-core (Team:Core) |
… src/core/server/integration_tests/ci_checks'
|
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management) |
| */ | ||
| export const getCurrentVirtualVersion = (type: SavedObjectsType): string => { | ||
| if (type.switchToModelVersionAt) { | ||
| if (type.modelVersions) { |
There was a problem hiding this comment.
One of the main changes in this PR is to assume we can relax the enforced switch to modelVersions. There is a risk that type owners continue to add migrations even though they are deprecated.
| validate({ type, kibanaVersion: '8.11.0' }) | ||
| ).toThrowErrorMatchingInlineSnapshot( | ||
| `"Migration for type foo for version 8.10.0 registered after switchToModelVersionAt (8.9.0)"` | ||
| `"Migration for type foo for version 8.11.0 registered after globalSwitchToModelVersionAt (8.10.0)"` |
There was a problem hiding this comment.
Validation still throws an error, so we should catch situations where new migrations are added.
TinaHeiligers
left a comment
There was a problem hiding this comment.
self review
kdelemme
left a comment
There was a problem hiding this comment.
ux logs management code change lgtm!
💚 Build Succeeded
Metrics [docs]Unknown metric groupsAPI count
References to deprecated APIs
History
|
|
Starting backport for target branches: 9.0 https://github.com/elastic/kibana/actions/runs/14682376211 |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
## Summary Part of elastic#201807 **Strategy**: This PR assumes that `switchToModelVersionAt` has done it’s job and all SO type owners now use `modelVersions` instead of `migrations`. It takes the approach of trusting that this is fine™ **Changes in this PR:** - `switchToModelVersionAt` removed from all registered saved objects, so their mapping hash had to be updated. - Implements `globalSwitchToModelVersionAt` directly in `version_map` - Updates logic to prioritize `modelVersions` over `migrations` in `getLatestMappingsModelVersion`, that previously relied on `switchToModelVersionAt` being set - Updates unit tests to match the refactored logic - Updates snapshots - Adapts SO types registered in the **cases**, **SLO** and **encrypted saved objects** plugins. - Updates docs **Risks**: - Plugin developers disregard the deprecated status of `migrations` and introduce new versions, which will not be BWC. Saved object type hash changes will notify the core team for a code owner review. The core team needs to investigate the related changes and provide feedback. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
|
backport PR: #219329 |
…#219329) # Backport This will backport the following commits from `main` to `9.0`: - [Removed switchToModelVersionAt from SO API definition (#219029)](#219029) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Christiane (Tina) Heiligers","email":"christiane.heiligers@elastic.co"},"sourceCommit":{"committedDate":"2025-04-26T15:00:09Z","message":"Removed switchToModelVersionAt from SO API definition (#219029)\n\n## Summary\nPart of https://github.com/elastic/kibana/issues/201807\n\n**Strategy**:\nThis PR assumes that `switchToModelVersionAt` has done it’s job and all\nSO type owners now use `modelVersions` instead of `migrations`.\nIt takes the approach of trusting that this is fine™\n\n**Changes in this PR:**\n- `switchToModelVersionAt` removed from all registered saved objects, so\ntheir mapping hash had to be updated.\n- Implements `globalSwitchToModelVersionAt` directly in `version_map`\n- Updates logic to prioritize `modelVersions` over `migrations` in\n`getLatestMappingsModelVersion`, that previously relied on\n`switchToModelVersionAt` being set\n- Updates unit tests to match the refactored logic\n- Updates snapshots\n- Adapts SO types registered in the **cases**, **SLO** and **encrypted\nsaved objects** plugins.\n- Updates docs\n\n**Risks**:\n- Plugin developers disregard the deprecated status of `migrations` and\nintroduce new versions, which will not be BWC. Saved object type hash\nchanges will notify the core team for a code owner review. The core team\nneeds to investigate the related changes and provide feedback.\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"19c017a111ff79155b2e7f94abe7601efd3d86c2","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Core","Feature:Saved Objects","technical debt","release_note:skip","backport:prev-minor","Team:obs-ux-management","v9.1.0"],"title":"Removed switchToModelVersionAt from SO API definition","number":219029,"url":"https://github.com/elastic/kibana/pull/219029","mergeCommit":{"message":"Removed switchToModelVersionAt from SO API definition (#219029)\n\n## Summary\nPart of https://github.com/elastic/kibana/issues/201807\n\n**Strategy**:\nThis PR assumes that `switchToModelVersionAt` has done it’s job and all\nSO type owners now use `modelVersions` instead of `migrations`.\nIt takes the approach of trusting that this is fine™\n\n**Changes in this PR:**\n- `switchToModelVersionAt` removed from all registered saved objects, so\ntheir mapping hash had to be updated.\n- Implements `globalSwitchToModelVersionAt` directly in `version_map`\n- Updates logic to prioritize `modelVersions` over `migrations` in\n`getLatestMappingsModelVersion`, that previously relied on\n`switchToModelVersionAt` being set\n- Updates unit tests to match the refactored logic\n- Updates snapshots\n- Adapts SO types registered in the **cases**, **SLO** and **encrypted\nsaved objects** plugins.\n- Updates docs\n\n**Risks**:\n- Plugin developers disregard the deprecated status of `migrations` and\nintroduce new versions, which will not be BWC. Saved object type hash\nchanges will notify the core team for a code owner review. The core team\nneeds to investigate the related changes and provide feedback.\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"19c017a111ff79155b2e7f94abe7601efd3d86c2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/219029","number":219029,"mergeCommit":{"message":"Removed switchToModelVersionAt from SO API definition (#219029)\n\n## Summary\nPart of https://github.com/elastic/kibana/issues/201807\n\n**Strategy**:\nThis PR assumes that `switchToModelVersionAt` has done it’s job and all\nSO type owners now use `modelVersions` instead of `migrations`.\nIt takes the approach of trusting that this is fine™\n\n**Changes in this PR:**\n- `switchToModelVersionAt` removed from all registered saved objects, so\ntheir mapping hash had to be updated.\n- Implements `globalSwitchToModelVersionAt` directly in `version_map`\n- Updates logic to prioritize `modelVersions` over `migrations` in\n`getLatestMappingsModelVersion`, that previously relied on\n`switchToModelVersionAt` being set\n- Updates unit tests to match the refactored logic\n- Updates snapshots\n- Adapts SO types registered in the **cases**, **SLO** and **encrypted\nsaved objects** plugins.\n- Updates docs\n\n**Risks**:\n- Plugin developers disregard the deprecated status of `migrations` and\nintroduce new versions, which will not be BWC. Saved object type hash\nchanges will notify the core team for a code owner review. The core team\nneeds to investigate the related changes and provide feedback.\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"19c017a111ff79155b2e7f94abe7601efd3d86c2"}}]}] BACKPORT--> --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…" (#220063) This reverts commit 19c017a. ## Summary Reverts #219029 as it is causing SO Migration crashes in our QA environment. It can also be tested locally on `main`: 1. On a previous version (`git checkout deploy@1745820726`) 2. Start ES: `yarn es serverless --projectType es` 3. Start Kibana: `yarn start --serverless=es` 4. Wait for migrations to complete 5. Stop Kibana 6. Move back to `main` 7. Start Kibana: `yarn start --serverless=es` 8. Wait for the error On `main`, we can see that Kibana fails to start with the fatal error: ``` FATAL Error: Unable to complete saved object migrations for the [.kibana_alerting_cases] index: Model version conflict: inconsistent higher/lower versions ``` In this branch, it starts successfully.
…" (#220059) Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…c#219029)" (elastic#220059) Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 75c0f95) # Conflicts: # src/core/server/integration_tests/ci_checks/saved_objects/check_registered_types.test.ts
…219029)" (#220059) (#220259) # Backport This will backport the following commits from `main` to `9.0`: - [Revert "Removed switchToModelVersionAt from SO API definition (#219029)" (#220059)](#220059) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Alejandro Fernández Haro","email":"alejandro.haro@elastic.co"},"sourceCommit":{"committedDate":"2025-05-05T14:01:45Z","message":"Revert \"Removed switchToModelVersionAt from SO API definition (#219029)\" (#220059)\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"75c0f9543acc80e0d875e908f8c4570581393e20","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-minor","Team:obs-ux-management","v9.1.0"],"title":"Revert \"Removed switchToModelVersionAt from SO API definition (#219029)\"","number":220059,"url":"https://github.com/elastic/kibana/pull/220059","mergeCommit":{"message":"Revert \"Removed switchToModelVersionAt from SO API definition (#219029)\" (#220059)\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"75c0f9543acc80e0d875e908f8c4570581393e20"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/220059","number":220059,"mergeCommit":{"message":"Revert \"Removed switchToModelVersionAt from SO API definition (#219029)\" (#220059)\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"75c0f9543acc80e0d875e908f8c4570581393e20"}}]}] BACKPORT-->
## Summary Part of elastic#201807 **Strategy**: This PR assumes that `switchToModelVersionAt` has done it’s job and all SO type owners now use `modelVersions` instead of `migrations`. It takes the approach of trusting that this is fine™ **Changes in this PR:** - `switchToModelVersionAt` removed from all registered saved objects, so their mapping hash had to be updated. - Implements `globalSwitchToModelVersionAt` directly in `version_map` - Updates logic to prioritize `modelVersions` over `migrations` in `getLatestMappingsModelVersion`, that previously relied on `switchToModelVersionAt` being set - Updates unit tests to match the refactored logic - Updates snapshots - Adapts SO types registered in the **cases**, **SLO** and **encrypted saved objects** plugins. - Updates docs **Risks**: - Plugin developers disregard the deprecated status of `migrations` and introduce new versions, which will not be BWC. Saved object type hash changes will notify the core team for a code owner review. The core team needs to investigate the related changes and provide feedback. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…c#219029)" (elastic#220059) Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…c#219029)" (elastic#220059) Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…20985) ## Summary * Address #217145 - Put in place a check to ensure we're upgrading from Kibana 8.18.0 or newer. * Address #220521 - New attempt at removing the `switchToModelVersionAt` property, inspired on #219029. The previous attempt caused a regression: index meta information started storing _modelVersions_ that were older than the previously stored ones, which were defaulting to 10.0.0 for SO types that did NOT define `modelVersions`. This was due to the removal of the applyTypeDefaults, which was ensuring all SOs had the `switchToModelVersionAt` property set. This flag was then used by `src/core/packages/saved-objects/base-server-internal/src/model_version/version_map.ts` to determine whether to use `modelVersions` or the legacy `migrations` property in order to determine the latest model version for a given type. When removing the `switchToModelVersionAt` flag (and its default backfill), the logic started defaulting to the latest `migrations` version for those SO types that were not defining any `modelVersion`, resulting in older versions that those stored in the SO indices. This caused incident https://elasticco.atlassian.net/browse/INC-3818. This regression has been shipped in 9.0.0 (the PR was [backported](#219329)), so in top of the cleanup, we now need to address #220521 to ensure a smooth transition _OnPrem => Serverless_. --------- Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…astic#220985) ## Summary * Address elastic#217145 - Put in place a check to ensure we're upgrading from Kibana 8.18.0 or newer. * Address elastic#220521 - New attempt at removing the `switchToModelVersionAt` property, inspired on elastic#219029. The previous attempt caused a regression: index meta information started storing _modelVersions_ that were older than the previously stored ones, which were defaulting to 10.0.0 for SO types that did NOT define `modelVersions`. This was due to the removal of the applyTypeDefaults, which was ensuring all SOs had the `switchToModelVersionAt` property set. This flag was then used by `src/core/packages/saved-objects/base-server-internal/src/model_version/version_map.ts` to determine whether to use `modelVersions` or the legacy `migrations` property in order to determine the latest model version for a given type. When removing the `switchToModelVersionAt` flag (and its default backfill), the logic started defaulting to the latest `migrations` version for those SO types that were not defining any `modelVersion`, resulting in older versions that those stored in the SO indices. This caused incident https://elasticco.atlassian.net/browse/INC-3818. This regression has been shipped in 9.0.0 (the PR was [backported](elastic#219329)), so in top of the cleanup, we now need to address elastic#220521 to ensure a smooth transition _OnPrem => Serverless_. --------- Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Summary
Part of #201807
Strategy:
This PR assumes that
switchToModelVersionAthas done it’s job and all SO type owners now usemodelVersionsinstead ofmigrations.It takes the approach of trusting that this is fine™
Changes in this PR:
switchToModelVersionAtremoved from all registered saved objects, so their mapping hash had to be updated.globalSwitchToModelVersionAtdirectly inversion_mapmodelVersionsovermigrationsingetLatestMappingsModelVersion, that previously relied onswitchToModelVersionAtbeing setRisks:
- Plugin developers disregard the deprecated status of
migrationsand introduce new versions, which will not be BWC. Saved object type hash changes will notify the core team for a code owner review. The core team needs to investigate the related changes and provide feedback.Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:*label is applied per the guidelines